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ABSTRACT 

The capacity of the North Central Association 
Commission on Secondary Schools self-study and team-visit program to 
effect change in member secondary schools was studied. Change is 
defined as the rate of implementation of recommendations. As many 
relevant variables as possible were identified and their 
interrelationship was investigated. A systems analysis framework was 
employed to identify and define these variables. They vere organize* 
ihto three categories: recommendation attributes, existing 
characteristics cf schools, and process variables. Data gathered from 
a represent at ive sample of c 7 schools was analyzed. Eactor analysis 
revealed a fourth category of variables, faculty and community 
attitudes toward and action taken on recommendations. The correlation 
coefficient achieved (.84) indicated that some of the system of 
variables related to change as measured by the implementation of 
recommendations had been determined. The aultioie regression equation 
a allowed an enumeration of the most important variables affectino 
implementation. The practical implications of this study for the 
evaluation process are discussed in some detail, (OR) 
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ACCREDITATION EVALUATION AND INSTITUTIONAL CHANGE 1 

The objective of this study* was to determine the effectiveness 

of the required North Central Association self study and team visit 

program in bringing about change in member secondary schools. Change 

was defined as the rate of implementation of recommendations. As 

many variables as possible were identified which are presumed to be 
* 

related to the implementation of recommendations. The interrelation- 
ship of these variables was also investigated. 

The Evaluation Process 

ii».i. » i«i» i» » . i « ■ « ■ ^ ■ 1 ■ — 

The North Central Association Commission on Se^bndary Schools 

requires each member secondary school to conduct a program of self 

evaluation every seven years. The self study program involves the 

appointment of faculty committees which over the course of as much as 

a year's time evaluate the existing school program and make recommenda 

tions for improvements in the future. The culmination of this 

activity is a visitation of usually three days by a team of "experts* 

* 

who build on the self evaluation and produce a document which outlines 
strengths and weaknesses of the school and also makes recommendations 
for school improvement. * 

Questions to be Answered 

The aim of .this study was to determine the total system of 
variables related to the implementation of recommendations. It was 
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assumed that change in the member schools/ as defined by the percent 
of implementation of the recommendations, would be the result of many 
factors. 

The test of this assumption required the identification and 
measurement of all possible variables related to implementation of 
recommendations. The first key question to be answered, therefore, 
was, "What variables are related to implementation of recommendations?" 

Utilizing a systems analysis framework, the research team 
members attempted cooperatively to identify and define the related 
variables.' The procedure employed was based on our (1) general 
knowledge of social systems and change (2) experimental knowledge of 
the evaluation process, and (3) study of related research on the 
evaluation process, 

It was also possible to logically organize the variables into 
three categories, These were: (1) recommendation attributes, (2) 

existing characteristics of the schools, and (3) process variables. 
"Recommendation attributes" included such items as the cost of 
implementing a recommendation , and the origin of the recommendation. 
"Existing characteristics of schools" included the relatively fixed 
characteristics of the school** over which North Central Association 
has little or no control, such as per pupil expenditure. "Process 
variables" included aspects of the evaluation process, such as 
duration of the self study, as well as attitudes of staff members toward 

, fc * 

the evaluation process. 

With these categories in mind, the second question to be 
answered was, "Will the relationship between the rate of implementation 




3 



of recommendations and given variables within a category persist after 
accounting for the effects of other categories of variables?" As an 
example, after analysis of the data it should be possible to say: 

(1) the evaluation process is most or least effective in bringing 
about change in schools with certain characteristics, e.g., in schools 
with high or low per pupil expenditures; (2) the process is most or 
least effective in schools where the evaluation process embodies certain 
characteristics, e.g., where the self study took place over a short 
versus a longer period of time; (3) the percent of implementation or 
recommendations of a given type was greater than for other types, e.g., . 
costly recommendations were implemented less often than less costly 
ones, and; (4) after accounting for the effects of one or two categories 
of variables, a given relationship still holds. For instance, after 
accounting for the effect of fixed characteristics, such as per pupil 
expenditure, and after accounting for the effect of attributes of 
the recommendations, such as cost of implementing the recommendations, 
process variables, such as duration of the self study, are signifi- 
cantly related to the percent of implementation of recommendations in 
the schools. 



Statement of Hypotheses 

% 

Although a number of variables were investigated, two major 
hypotheses were tested. The variables used may be thought of as sub- 
hypotheses under each category of the second major hypothesis. The 
major hypotheses were: 
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H.l A model of the system of variables associated with impTementation 
of recommendations can be established and some indication of the rela- 
tive importance of variables can be determined. 

H.2 The implementation of recommendations is related to several 
variables. When the variables are assigned to categories, the 
relationship of given variables to implementation will persist in- 
dependently of the effects of other categories of variables. The 
categories of variables are: 

1. Recommendation attributes 

2. Existing characteristics of schools 

3. Process variables 

The categories into which the variables wc. e placed are not 
only of importance conceptually. Of equal importance are statistical 
considerations. If categories of variables, or representative 
variables for these categories, are to be used as "controls*' in 
partial correlations it is important to know the relationships of 
these variables and categories. Ideally the categories should be 
substantially orthogonal. At the same time the variables within the 
categories should be fairly highly intercorrelated if the category is 
to be economically represented by some representative variables. 

To meet these criteria, the variables were factor analyzed to check 
the content of the categories which had been conceptualized logically. 

The total of the variables available for factor analysis 
resulted from the administration of questionnaires tc the faculties 
of the 5? schools in the sample and from the collection of various 
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data about these schools. These sources yielded a pool of 186 variables. 
However, since the number of variables should be less than the 
number of subjects in factor analysis, only those variables which 
correlated at .20 or higher with implementation of recommendations 
were used. This offered a means of selection of the most important 
variables and reduced the number used to 49. 

In general, the factors identified were not in serious discord 
with the definitions of categories previously established. The factor 
analysis did result in the realignment of some variables within 
categories. However, the main result was the addition of another 
category of variables having to do with faculty and community attitudes 
toward and actions taken on individual recommendations. These attitudes 
were clearly different from the generalized attitudes toward the self 
study or team visit. Temporally, they have to do with events after 
the self study and team visit wet a completed. Also, because of the 
location of these items on the questionnaire they were somewhat suspect 
of an interpretation of causality. For example, a person having just 
indicated that a recommendation had been implemented, would be 
inclined to infer, even if he did not know, that the principal, for 
instance, had supported the recommendation. For these reasons, and in 

i t 

order to be able to handle this set of variables separately, a new 
category was established. This fourth category was titled "Attitude 
Toward and Action on Recommendations." 
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Procedures 

The sample consisted of 57 schools which had conducted the 
self study about two years prior to this investigation. These schools 
were as representative as possible of the North Central Association 
membership. 4 

Because the implementation of recommendations was investigated 
quite closely, not all of the recommendations made for a school could 
be investigated. All of the recommendations made for each school 
were divided into categories of (1) Articulation and Coordination, 

(2) Curriculum, (3) Facilities, and (4) Personnel. A category of (5) 
Innovations was also used, though this category was not independent 
of the first four. Those recommendations selected for investigation 
were the four most important recommendations from each category, in 
the judgment of two members of the research team The percent of 
implementation of recommendations for each school was defined as 
those recommendations selected in the first four categories which were 
either completely or partially implemented. 

The uata were collected as follows: First, a research team 

member called on each school in the sample during which he interviewed 
the principal and pther faculty members, examined documentary evidence 
pertaining to implementation, administered a questionnaire to the 
superintendent, principal, and 10 teachers, and recorded information 
from the E valuat iv e Criteria and the North Central Association Annual 
Peport. Next, other available data regarding schools and states were 
extracted from several studies which had been conducted previously. 
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The data were analyzed in two ways using: (1) each recommend- 

ation as the unit of analysis, and (2) each school as the unit of 
analysis. The first analysis utilized chi square tests of significance 
to compare rates of implementation for various types of recommendations. 
The second analysis utilized primarily multiple regression. Repre- 
sentative variables were selected for each category of variable through 
a stepwise multiple regression procedure. These representative 
variables were used in the equation to determine whether a given 
relationship of variables persisted independently of the effects of 
variables in other cateogires. These partial correlational zero order 
correlations i and the multiple regression equation were used to 
determine relationships constituting the total system relating to 
implementation of recommendations. 

Results 

With respect to the recommendations selected for each school, 
when implementation consisted of both .complete and partial implemen- 
tation, the percentages o<: implementation by categories were: 
Articulation and Coordination, 61.5 percent? Curriculum, 53.6 percent? 
Facilities, 41.8 percent? Personnel, 57.8 percent? Total for the above, 
53.6 percent? Innovation (not an independent category), 44.6 percent. 
The mean number of recommendations per category per school was as 
follows: Articulation and Coordination, 14.5? Curriculum, 52.6? 

Facilities, 38,0; Personnel, 13.3? Innovations, 1.8? Total 116.5. 

The dependent or criterion variable consisted of the percent 
of selected recommendations implemented in each school. As indicated 
above, the mean percentage of these recommendations impletnented was 
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53,6. The range of the dependent variable was from 25 to 86, the 
standard deviation was 15,2, and a probit analysis indicated nc 
departure from normality. 

Two multiple regression equations were determined. Both 
utilized a forward step wise procedure to select the variables in 
the equation. The first equation was determined using all -49 variables 
which correlated at .20 or higher with the criterion as the possible 
independent variables. The second equation was determined using the 
12 representative variables as the possible independent variables. 

In both equations the same five variables were selected first. These 
five variables explained about 70 percent of the criterion variance 
(multiple correlation of ,84), The predictive model was therefore 
achieved as stated in the first major hypothesis. 5 

In the analysis by schools, the characteristics of the recom- 
mendations for each school were either averaged or percent scores 
determined as appropriate. Thus the same information was available, 
by and large, for this analysis and the analysis of recommendations. 

In only one case did the analysis by recommendations reveal a 
relationship that was not shown in the analysis by schools. The 
individual recommendation analysis showed that recommendations which 
originated with the visiting team were less likely to be implemented 

than v'ere those which arose previous to the team visit. However, 

> 

the percent of recommendations which originated with the visiting 
team when determined for each school failed to correlate significantly 
with the total percent of implementation for the school. The probable 
explanation is that this is a real relationship which was masked or 
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randomized in the school by school analysis. 

The second major hypothesis was also borne out. Using the 
categories established for the variables, representative variables 
were entered into the equation and partial correlations were deter- 
mined for the remaining variables, enabling relationships to be 
established which could be determined to persist even after accounting 
for the effects of other categories of variables. Findings will be 
enumerated below. The order of presentation, insofar as possible, 
follows the order of importance established by the multiple regression 
equation, 

1. Attitude toward and support of individual recom- 
mendations on the part of all of the faculty and 
particularly the principal is very important in 
the implementation of recommendations. Positive 
support by some part of the faculty is necessary 
for implementation. Those recommendations which 
are not implemented are not often resisted. 

Instead, it would seem they are not implemented 
because nobody pushes for them. 

2. Teacher and principal attitudes toward and per- 
ceptions of the total evaluation process are 
related to the percent of recommendations 
implemented. While principals focus on both 
the team visit and the self study, the teacher 
attitudes and perceptions concerning the team 
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visit are more important in relationship to 
total implementation. Other variables which 
show a positive relationship to implementation 
are teacher satisfaction with the self study 
instrument and teacher perception of the 
presence of a formal follow-up program. 

3. The cost of implementing recommendations is 
inversely related to the rate of implementation. 
Those recommendations which cost more will be 

' implemented less often. 

4. The kind of school is important in the success 
of the evaluation process as measured by the 
implementation of recommendations. There will 
be greater implementation of recommendations in 
schools which are less "sophisticated" as in- 
dicated by (1) the lack of a professional nego- 
tiation agreement with teachers' groups, par- 
ticularly the AFT, (2) a principal who has 
fewer semester hours in administration, and (3) 

a .greater percentage of students intending to go 
to a post secondary school other than college 
or junior college, 

5. There is greater implementation of those recom- 
mendations which originated previous to the 
team visit rather than with the visiting team. 
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There is also greater implementation of those 
recommendations which originated with the self 
study than of those which were is existence 
previous to the self study. 

6. There is greater implementation of recommendations 
in those schools in which greater importance was 
attributed to the role of the evaluation process 
in achieving implementation of those recommenda- 
tions implemented. 

Concluding Remarks 

The multiple correlation coefficient achieved (.84) gives in- 
dication that some of the system of variables related to change as 
measured by the implementation of recommendations has been determined. 
The same statistic indicates that all related variables have not been 
identified. Future research should be able to identify more related 
variables. In this study, process variables were not well identified 
because of the time lag, Research in which the process would be 
studied as it takes place should be able to better identify these 
variables. .... 

* . 

This study also seems to have practical implications for the 

evaluation process. Some of these points follow: 

The importance of support of; recommendations by the faculty 
in the implementation of recommendations may be noted. If the team 
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visit, is to be effective in bringing abput change, the team members 
should be concerned with support that they can enlist for recom- 
mendations and strategies for enlisting this support. Even more 
important, if a recommendation can be introduced at the level of t‘ e 
self study, it will have greater chances of success. All of this 
is in line with the concept of the importance of self determination 
and participation which underlies the whole evaluation process. 

The average number of innovative recommendations per school (1.8) 
indicates that the evaluation process has done little to introduce 
innovations. Rather, it would seem that many recommendations ar: a 
sort of bringing-up-to-standards type. If the evaluation process is 
used in the future as a vehicle to introduce new ideas and methods, 
the above considerations will become especially important. 

The role of the visiting team and the importance of the per- 
ception of teachers concerning the visiting team is important. 

Whether in those schools with higher percentages of implementation 
the visiting team performed more adequately or the teachers simply 
had less stringent or more realistic expectations cannot be determined 
from these data, Very likely both could be true to some extent. 

It would seem that North Central Association' should deal with this 
important aspect of the evaluation process by: (1) clarifying the 

role of the visiting team, especially as it concerns the classroom 
teacher, (2) working toward helping the faculty of the school under- 
stand this role and the relative importance of the self study and (3) 
helping in the selection and training of team members to enable them 
to do a more adequate job. 
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The Importance of a follow-up program within the school has 
been indicated; one that gets across to teachers. The responses 
of principals to the interview also showed that most principals would 
welcome a follow up small scale visit if this did not include the 
mandatory implementation of recommendations. The findings here would 
indicate that such a program should not be restricted to interaction 
with the administration. 

Finally, the finding that the evaluation process has greatest 
impact on the least sophisticated schools should be a matter of some 
concern, for the converse of this is that it has least impact on the 
more sophisticated schools. It may be argued either that the former 
need more help or that the latter are more impervious to the 
evaluation process. Probably both statements are true to some extent. 
Perhaps one factor limiting impact in big city and larger school 
districts is the lrck of any district wide approach which would 
coordinate the results of the individual school evaluations. While 
the lack of impact may result in part from a generalized attitude 
concerning North Central Association, thought should be given to 
making the evaluation process more meaningful for these more 
sophisticated schools. 
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FOOTNOTES 



3., This paper is based on the author's unpublished doctoral 
dissertation, "Accreditation Evaluation and Institutional 
Changes A study of the Implementation of Recommendations 
in Selected North Central Association Secondary Schools," 
Northwestern University, 1969, 

2. In the schools studied this process was based on the use 
of Evalua tive Cri ter ia, 1960 Edition , National Study of . 
Secondary^ School Evaluation (Washington, D.C. 1960) , and 
the High School Evaluation Guide , North Central Associa- 
tion of Colleges and Secondary Schools (Chicago, Illinois, 
1965) . 

3. The research team, consisted of: Gordon Cawelti, Executive 

Secretary of the N >rth Central Association Secondary School 
Commission? Larry Master, Indiana University; and Glen 
Meredith, University of Wisconsin? and the author. The 
present study was one part of a larger study. Larry Master 
and Glen Meredith collected some of the data used in the 
present study and conducted related studies on attitude 
about the evaluation process and teacher perceptions of 
changes resulting from the evaluation process. 

4. Some of the less populous states were under-represented, 

as were suburbs and small towns, and the average enrollment 
of the sample schools was higher than that of member schools. 
On the other hand, the proportion of schools under 500 
enrollment and the proportion of public and non-public 
schools was the same for the sample and the membership. 

The distribution and characteristics of the schools in the 
sample are shown in a table on the next page. 



ERIC 



DISTRIBUTION AND CHARACTERISTICS OF SCHOOLS IN THE SAMPLE 



: 




. Area 


No. • : 

Non- 1 


Avg, no. 
Students 


State 


No. 


1 


2 


3 


4 1 


.5 


Public ! 


Arkansas ■ 


3 




3 










• 1002 


Illinois 


9 


3 


1 


2 


1 


2 


4 


1559 


• 

Indiana 


6 


2 


3 






1 


1. 


1190 


Iowa . 


3 


• 


. 2 






1 


1 


1167 


Kansas. 


3 






1 


1 


1* 


1 


1026 


Michigan 


5 


1 


2 


1 


1 






1156 


•Minnesota 


4 




1 




2 


1 


J 


672 


Missouri 


3 


1 


2 










1585 


Ohio 


6 


1 


1 


2 


1 


1 


*• * 


1147 


Oklahoma 


3 


1 


2 










. 1205 


West Virginia 


3 




1 1 




1 


1 


1 


578 


Wisconsin 


9 


1 


4 ' 


* 


3 i 


1 


1 


850 


Total 


57 


10 


22 

^ 


k 


■10 


9 


6 


1126 



Notej Area numbers indicate 'V 

1 . City over 400,000 residents • . 

2 . . Community of 5 , 000 - 399 » 000 (not suburban) 

3 » Suburban - within close proximity to city of 
over 100,000 " 

4 , Small town of under 5 » 000 

5 .. Rural . • . ’ . 
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5. The equation is shown in the following table: 

FINAL MULTIPLE REGRESSION EQUATION WITH TOTAL 
IMPLEMENTATION OF RECOMMENDATIONS 
’ ' ■ AS CRITERION ■ 



Regression 

Coefficient 



Variable 



After Each Step 

R ^ 



151.19 
+19. TO 

-12.32 

+13.07 



+ .46 

-5.17 



(constant) 

Principal's action on 
’ recommendations 
Cost of recommendations 
Teacher's rating, evalua- 
tion process obtains 
support for needs 
Percent students, intend- 
ing post secondary schoc 
other than college' 
Presence of professional 



.569 


.324 


.701 


" -.492 ; 


.786 


00 

r-t 

* 


.1* ' 

.816 


. 666 


.836 


.698 



The selection of predictor variables which correlate highly 
with the criterion variable from a number of independent 
Variables is bound to capitalize on chance. If this same 
equation were employed on a nev; sample or on the total 
population, where different or no change factors were opera 
ting, the present change inflation of R2 would result in 
"shrinkage 1 * of when applied to the new sample. 




